Parser Evaluation: a Survey and a New Proposal

We present a critical overview of the state-of-the-art in parser evaluation methodologies and metrics. A discussion of their relative strengths and weaknesses motivates a new—and we claim more informative and generally applicable—technique of measuring parser accuracy, based on the use of grammatical relations. We conclude with some preliminary results of experiments in which we use this new scheme to evaluate a robust parser of English.